| 1. |
- Judisky, Anatoli, et al.
(author)
-
Minimax Confidence Intervals for Pointwise Nonparametric Regression Estimation
- 2009
-
In: Proceedings of the 15th IFAC Symposium on System Identification. - Linköping : Linköping University Electronic Press.
-
Reports (other academic/artistic)abstract
- We address a problem of estimation of an unknown regression function f at a given point x0 from noisy observations yi = f(xi)+ ei, ;i =1, ..., n. Here xi in ε Rk are observable regressors and (e_i) are normal i.i.d. (unobservable) disturbances. The problem is analyzed in the minimax framework, namely, we suppose that f belongs to some functional class F, such that its finite-dimensional cut Fn = {f(xi), f ε F, i =0, ..., n, } is a convex compact set. For an arbitrary fixed regression plan Xn =(x1;...;xn) we study minimax on Fn confidence intervals of affine estimators and construct an estimator which attains the minimax performance on the class of arbitrary estimators when the confidence level approaches 1.
|
|
| 2. |
- Nazin, Alexander, et al.
(author)
-
Direct Weight Optimization in Nonlinear Function Estimation and System Identification
- 2007
-
In: Proceedings of the 6th International Conference on System Identification and Control Problems (SICPRO '07). - Linköping : Linköping University Electronic Press.
-
Conference paper (peer-reviewed)abstract
- The Direct Weight Optimization (DWO) approach to estimating a regression function and its application to nonlinear system identification has been proposed and developed during the last few years by the authors. Computationally, the approach is typically reduced to a quadratic or conic programming and can be effectively realized. The obtained estimates demonstrate optimality or sub-optimality in a minimax sense w.r.t. the mean-square error criterion under weak design conditions. Here we describe the main ideas of the approach and represent an overview of the obtained results.
|
|
| 3. |
- Roll, Jacob, 1974-, et al.
(author)
-
A General Direct Weight Optimization Framework for Nonlinear System Identification
- 2005
-
In: Proceedings of the 16th IFAC World Congress. - Linköping : Linköping University Electronic Press. - 9783902661753 ; , s. 29-29
-
Conference paper (peer-reviewed)abstract
- The direct weight optimization (DWO) approach is a method for finding optimal function estimates via convex optimization, applicable to nonlinear system identification. In this paper, an extended version of the DWO approach is introduced. A general function class description --- which includes several important special cases --- is presented, and different examples are given. A general theorem about the principal shape of the weights is also proven.
|
|
| 4. |
- Roll, Jacob, et al.
(author)
-
A Non-Asymptotic Approach to Local Modelling
- 2002
-
In: Proceedings of the 4th Conference on Computer Science and Systems Engineering. - 0780375165 ; , s. 75-80
-
Conference paper (other academic/artistic)abstract
- Local models and methods construct function estimates or predictions from observations in a local neighborhood of the point of interest. The bandwidth, i.e., how large the local neighborhood should be, is often determined based on asymptotic analysis. In this paper, an alternative, non-asymptotic approach that minimizes a uniform upper bound on the mean square error for a linear estimate is proposed. It is shown, for the scalar case, that the solution is obtained from a quadratic program, and that it maintains many of the key features of the asymptotic approaches. Moreover, examples show that the proposed approach in some cases is superior to an asymptotically based local linear estimator.
|
|
| 5. |
- Roll, Jacob, et al.
(author)
-
Local Modelling of Nonlinear Dynamic Systems Using Direct Weight Optimization
- 2003
-
In: Proceedings of the 13th IFAC Symposium on System Identification. - Linköping : Linköping University Electronic Press. - 0080437095 ; , s. 1554-1559
-
Conference paper (peer-reviewed)abstract
- Local models and methods construct function estimates or predictions from observations in a local neighborhood of the point of interest. The bandwidth, i.e., how large the local neighborhood should be, is often determined based on asymptotic analysis. In this paper, an alternative, non-asymptotic approach that minimizes a uniform upper bound on the mean square error for a linear estimate is used. It is shown that the estimator is obtained from a quadratic program, that an automatic bandwidth selection is obtained, and that the approach can be seen as a local version of fitting affine models to data. Finally, the approach is applied to two benchmark systems.
|
|
| 6. |
- Roll, Jacob, et al.
(author)
-
Local Modelling with A Priori Known Bounds Using Direct Weight Optimization
- 2003
-
In: Proceedings of the 2003 European Control Conference. - Linköping : Linköping University Electronic Press.
-
Conference paper (peer-reviewed)abstract
- In local modelling, function estimates are computed from observations in a local neighborhood of the point of interest. A central question is how to choose the size of the neighborhood. Often this question has been tackled using asymptotic (in the number of observations) arguments. The recently introduced direct weight optimization approach is a non-asymptotic approach, minimizing an upper bound on the mean squared error. In this paper the approach is extended to also take a priori known bounds on the function and its derivative into account. It is shown that the result will sometimes, but not always, be improved by this information. The proposed approach can be applied, e.g., to prediction of nonlinear dynamic systems and model predictive control.
|
|
| 7. |
- Roll, Jacob, 1974-, et al.
(author)
-
Nonlinear System Identification Via Direct Weight Optimization
- 2005
-
In: Automatica. - Linköping : Elsevier. - 0005-1098 .- 1873-2836. ; 41:3, s. 475-490
-
Journal article (peer-reviewed)abstract
- A general framework for estimating nonlinear functions and systems is described and analyzed in this paper. Identification of a system is seen as estimation of a predictor function. The considered predictor function estimate at a particular point is defined to be affine in the observed outputs, and the estimate is defined by the weights in this expression. For each given point, the maximal mean-square error (or an upper bound) of the function estimate over a class of possible true functions is minimized with respect to the weights, which is a convex optimization problem. This gives different types of algorithms depending on the chosen function class. It is shown how the classical linear least squares is obtained as a special case and how unknown-but-bounded disturbances can be handled. Most of the paper deals with the method applied to locally smooth predictor functions. It is shown how this leads to local estimators with a finite bandwidth, meaning that only observations in a neighborhood of the target point will be used in the estimate. The size of this neighborhood (the bandwidth) is automatically computed and reflects the noise level in the data and the smoothness priors. The approach is applied to a number of dynamical systems to illustrate its potential.
|
|
| 8. |
- Iouditski, Anatoli, et al.
(author)
-
Adaptive DWO Estimator of a Regression Function
- 2007
-
Reports (other academic/artistic)abstract
- We address a problem of non-parametric estimation of an unknown regression function f : [-1/2, 1/2] → R at a fixed point x0 € (-1/2, 1/2) on the basis of observations (xi, yi), i = 1,..,n such that yi = f(xi) + ei, where ei ~ N(0, σ2) is unobservable, Gaussian i.i.d. random noise and xi € [-1/2, 1/2] are given design points. Recently, the Direct Weight Optimization (DWO) method has been proposed to solve a problem of such kind. The properties of the method have been studied for the case when the unknown function f is continuously differentiable with Lipschitz constant L. The minimax optimality and adaptivity with respect to the design have been established for the resulting estimator. However, in order to implement the approach, both L and σ are to be known. The subject of the submission is the study of an adaptive version of DWO estimator which uses a data-driven choice of the method parameter L.
|
|
| 9. |
- Ljung, Lennart, 1946-, et al.
(author)
-
Adaptive DWO Estimator of a Regression Function
- 2004
-
In: Proceedings of the 2004 IFAC Symposium on Nonlinear Control Systems.
-
Conference paper (peer-reviewed)abstract
- We address a problem of non-parametric estimation of an unknown regression function f : [-1/2, 1/2] → R at a fixed point x0 € (-1/2, 1/2) on the basis of observations (xi, yi), i = 1,..,n such that yi = f(xi) + ei, where ei ~ N(0, σ2) is unobservable, Gaussian i.i.d. random noise and xi € [-1/2, 1/2] are given design points. Recently, the Direct Weight Optimization (DWO) method has been proposed to solve a problem of such kind. The properties of the method have been studied for the case when the unknown function f is continuously differentiable with Lipschitz constant L. The minimax optimality and adaptivity with respect to the design have been established for the resulting estimator. However, in order to implement the approach, both L and σ are to be known. The subject of the submission is the study of an adaptive version of DWO estimator which uses a data-driven choice of the method parameter L.
|
|
| 10. |
- Ljung, Lennart, 1946-, et al.
(author)
-
Asymptotically Optimal Smoothing of Averaged LMS for Regression Parameter Tracking
- 2002
-
In: Proceedings of the 15th IFAC Congress. - 9783902661746 ; , s. 436-436
-
Conference paper (peer-reviewed)abstract
- The sequence of estimates formed by the LMS algorithm for a standard linear regression estimation problem is considered. It is known since earlier that smoothing these estimates by simple averaging will lead to, asymptotically, the recursive least squares algorithm. In thi spaper it is first shown that smoothing the LMS estimates using amatrix updating will lead to smoothed estimates with optimal tracking properties, also in the case the true parameters are slowly changing as a random walk. The choice of smoothing matrix should be tailored to theproperties of the random walk. Second, it is shown that the same accuracy can be obtained also for a modified algorithm, SLAMS, which is based on averages and requires much less computations.
|
|
